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Abstract. Different mathematical models of recognition processes are known. 
In present paper we consider a recognizing algorithm as an oracle computation 
on Turing machine [Rogers]. Such point of view seems to be useful in pattern 
recognition as well as in recursion theory. Use of recursion theory in pattern 
recognition shows connection between a recognition algorithm comparison problem 
and complexity problems of oracle computation [Bulitko] . That is because in many 
cases we can take into account only the number of sign computations or in other 
words volume of oracle information needed. Therefore, the problem of recognition 
algorithm preference can be formulated as a complexity optimization problem 
of oracle computation. Furthermore, introducing a certain "natural" preference 
relation on a set of recognition algorithms we discover it to be nontransitive. This 
relates to the well known nontransitivity paradox in probability theory [Szekely]. 
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Most of our notation follows [Rogers], also r.e. means recursively enumerable. 

Let (-Pi) igN be a system of one-place predicates (basic signs) defined on a pattern 
set U. We assume (Va, b G U) [a ^ b =>- 3i [P»(a) 7^ Pj(6)]]. Thus it is possible to 
represent every pattern a E U as a subset a C N : a — {i | Pi (a)}. Also D = a ^ 
a = D. So an image B, which is a set of patterns from U, can be represented as a 
subset of 2 N . Let S be a set of all images. So the recognition problem for given, in 
general partial, function r : N — » S and given pattern set U can be formulated as 
to find such z that for every pattern x £ U: 
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1, if x G r (i) and r (i) is defined 
ip x z (i) — I 0, if x t (i) and r (i) is defined 
undefined, if t (i) is undefined 

ff such z exists then we say the recognition problem is solvable for triple (U, r, 

Definition. We use for U if 

(1) Vx G U [\x\ < oo]; 

(2) {p | Dp = rr G £/} is recursive. 

Definition, r is called computable ^ (3 partial recursive 7) Vi [(r(i) 7(1) |) 
& ( 7 (i) w 7(i) = {p I -Dp = 5, x G r(i)})]. 
The following proposition can be easily proved. 
Proposition. 

(1) There is a triple (if, r, (-Pi) i6N ) swc/i that r zs tota/ 7 r(i) is countable for every 
i, and the recognition problem for (U,r, (-Pj) igN ) is unsolvahle. 



(2) For given non-empty set U* there is a computable total r such that the recog- 
nition problem for (ll^ , r, (Pi) ieN j is unsolvahle. 

We should consider unsolvability of a recognition problem as an indication of 
non-adequacy of the basic sign system to image set S and function r. This means 
any algorithm z makes mistakes. However some of the algorithms might make more 
mistakes than others. The following is a formalization of that. 

Let us consider (iff, r, (Pi) iem j , where r is computable. Define B = {{k,i) \ 

k G W 7 (j) (i.e Dk G r(i))}. B is r.e. If B is not recursive then (ll^, r, (Pi) ie ^j is 
unsolvable, however every algorithm z has a finite correctness domain [0, n] equal to 
such a maximal initial segment of N that (Vj G [0, n}) {[j — (k,i) G B =>• <pf k (i) = 1] 
& [j = </c, i> £B=^?*(i) = 0]}. 

So it is natural to choose new signs on patterns from U* such that there is new 
recognition algorithm ip z i with correctness domain strictly including correctness do- 
main of ip z . Definitely we have to keep doing that all the time and use new informa- 
tion about the images. It is possible to estimate the volume of the information using 
its Kolmogorov complexity [Loveland]. Namely given computable r the volume of 
information sufficient to define new signs P* and construct recognition algorithm for 
(lJf,T, (P*) ieN j ,that has correctness domain with length n, grows not faster than 
Inn. 
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Usually an image includes a given pattern along with its various modifications. 
They can be got by effective transformations of the pattern. In that more general 
case let B — ( U R q ) f| W e where 6 C N, R q is a recursive set such that C Rq = ip g ( q y, 

and mining > h(q) for every q G Q; here g, h are total recursive functions and h is 
not a decreasing function. KRq is resolving complexity of G [Loveland]. 

Proposition. Taking into account above made assumptions we do not need more 
than Inn + KR^ih^in)) of information about W e © G to construct new signs P* 
such that the recognition problem (Iff , (r(i )), (P*) n6N ) is solvable. 

In a training process a hypothesis about teacher's signs can be transformed into a 
hypothesis about enumeration operator. So a recognition algorithm can be found us- 
ing the Kleene recursion theorem. The last proposition allows to estimate frequency 
and volume of operator hypothesis changes. 

So we can introduce time of the training. It would be interesting to introduce 
time in a recognition process. We propose the following method to do that. 

As well known, there is a passage from r.e. set W z defining partial recursive in 
A function tpy- to tree diagram T z of the Turing machine computation with oracle 
A. Tree T z can be equivalently represented as r.e. set Wf( z ) containing the tree's 
terminated branches. A passage from W z to Wf( z ) is called regularization. It can 
be done in such a way that / will be recursive and some additional conditions 
for Wft z \ will be satisfied. Notation (x,y,u,v) G 1 Wf( z \ means there is a branch 
(x, n\O\n<i0~i ■ ■ ■ rikCTk, y) belonging to Wf( z ) such that D u = {n t \ a t — 1}& D v — 
{n t | a t = 0}. W z>t denotes result of enumeration of W z by algorithm z after t steps 
done. 

Proposition. There is a recursive function f such thatipf(x) = y =3- ip^^x) = y; 



VW y 



where Wfii z \ = {(x,y,u,v) 



U (D u U D v ) D U (A. U D v ) 

_{x,y,u,v)eWz,t (x,y,u,v)£W f x (z),t 

(x,y,u,v) G 1 W f{z) }. 

So using this approach we can represent recognition algorithms as trees. Vertexes 
of these trees contain the signs to be computed. The end of a terminated branch of 
a tree contains (pf (i) (i.e. it indicates if A G r (i) or not). 

In practice image system S and patterns are finite and every a, G S is a recursive 
subset of Iff. In that case the recognition problem for (U^,(ai,... ,ak), n en) is 
definitely solvable however there is a complexity problem. Also we should mention 
the best algorithm problem comes up if we introduce a certain preference relation 
on the set of recognition algorithms. In order to consider that we assume sign 
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computation is an easy process (i.e. any sign computation takes one time unit). 

Assume c^n a,- = if % ^ j; i, j G {1, . . . , k} and U f = Lla*. Then evidently there 

« 

is a recursive function q such that (\/x G t/-^ y^(0) = i ^ tp x z (i) = lj . Under 
these assumptions one tree represents one recognition algorithm and any branch of 
such tree ends. 

We call branch length (quantity of arcs) recognition time for pattern x by algo- 
rithm A and write it T (A,x). Recognition time for a set M C by algorithm A 
is T (A, M) = maxT(A,x). 

Let R be a recognition algorithm set. Algorithm A defeats algorithm B recognition 
pattern x iff T (A, x) <T{B,x). Define V (A, B) = \{x G U \ T{A,x) < T(B,x)}\. 

Consider a situation when algorithms A and B are in process of recognizing ran- 
dom pattern sequences. Let n(t), 1 < t < n be a random pattern sequence. Let 
v (x, t) be the probability of event n (t) = x. So the mathematical expectation 
m (A, B, n) of the number of members in n on which A defeats B can be expressed 
as follows: 

E E v(x,t)sg(T(B,x)^T(A,x)) 

x£Uf t=l 

, , s I l,o>0 , f a — b, a > b 

where sg (a) = ^ ,a-6= j ^ " . 



m(A,B,n) _ V(A,B) 
n ~ \uf I 



and therefore rec- 



Assuming t> (x, t) = const we get (Vn > 1) 

ognizing the random pattern sequence algorithm A wins more times in average 
than algorithm B iff V(A, B) > V(B,A). So we can introduce a preference re- 
lation on R : we say algorithm A is better than algorithm B (write A <C B) iff 
V (A, 5) > V (5, A). We call algorithm A equivalent to B iff V (A, B) = V (B, A). 
It is clear that any two algorithms are comparable and the preference relation seems 
to be natural. Consider an example demonstrating nontransitivity of the introduced 
preference relation. 

Let us define 03 = {0,1}, so 03 9 = jii^ . . . iz \ i s G 03, sGl,9| and {i\ . . . 

<"m\—\ 

03 i-m-i+i ■ ■ ■ i m k -i^im k +i • • • 29} means {i 

1 • • • 'mi-l'mimi+l • • • — •••29 

im s G 03; s — 1, k; < k < 9} C 03 9 . Here and below a, b means {a, . . . , b}. 

Now define U f C 03 9 as U f = a U a t U a 2 U a 3 where a = {103030103001}, 
a x = {010300110303}, a 2 = {001103030 1 03}, a 3 = {000000000}. Obviously |a„| = 
l a i| — |cK2 1 = 8, | CK3 j = 1, Iff = 25. Set S = {ao, oti, a 2 , 0:3}. 

Consider nine predicates on Iff defined as (x) — ik, k — 1, 9. Also we consider 
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Fx 




Figure 1. Structure of recognition algorithms 



Pi Pa Pi 




Figure 2. Recognition algorithms A, B, C 

a set 1Z of recognition algorithms which are binary trees in the form shown on [Fig. 
!]■ 

There Fj G jPfc | = 1, 9| , a>i q £ ja s | s = 0, 3|. We call Fi the first predicate in 
the algorithm or predicate in the root. 

Every pattern x corresponds to a branch in tree of algorithm A from the root to 
a leaf. The leaf is output of algorithm A recognizing pattern x. We assume jump 
to the left subnode of node Fj iff Fj (x) is true. 

Finally consider three algorithms shown on [Fig. 2]. 

[Tab. 1] shows their recognition time. 

Theorem 1. 

(1) A<B,B< C, C< A. Therefore the preference relation is nontransitive. 
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M\Y 


A 


B 


C 


a 


1 


2 


3 


«i 


2 


3 


1 


«2 


3 


1 


2 


«3 


3 


3 


3 



TABLE 1. Recognition time of algorithms A, B, C 



(2) There is no such X G K \ {A, B, C} that (A < X) V (B < X) V (C < A"). 
Proof. 

The first statement follows from [Tab. 1]. 

Consider the second statement. Let X be an arbitrary recognition algorithm. We 
have to investigate two cases: 

(1) Mx G U f [T(X,x) > 2]; 

(2) 3x G U f [T(X,x) = lj. 

For the first case we will go through all the predicates which can be situated in 
the root of the tree of X. Those predicates are P 2 , P3, P5, Pq, Ps, P9 because Pi, 
P 4 , P 7 separate an entire image. We will pay attention only to P 2 because the other 
cases are similar. 

So suppose P 2 is the first predicate computed in X. It will "divide" all the patterns 
into two sets: jla^^i} an d {|«o? a 2, ^3}- Here and below |a is a subset of a 
such that 2q, | — £ ja;o| - It is clear we can easily split the first of those two sets. 
However we cannot split the second set by one predicate because the second set 
contains patterns belonging to the three images. So we get two sub cases: 

(1) we separate patterns belonging to «o from the second set and split 0:2, «3 by 
another predicate. The scheme on [Fig. 3] shows that sub case. 

(2) we separate a 2 from the second set first and then split ^a and a 3 [see Fig. 

4]- 

However comparison of these algorithms with A, B, C shows that the first one is 
equivalent to B but worse than A and C . The second algorithm is equivalent to 
A but worse than B and C . Obviously the other algorithms with P 2 as the first 
predicate (i.e. in the root of tree) are even worse. 

Consider the second case. Obviously we have to put P2 or P4 or Pj in the root if 
we want to have an image separated (recognized) in one time unit. So we have all 
patterns belonging to one image if the first predicate is true and patterns from the 
other three images otherwise. The scheme on [Fig. 5] illustrates that. 
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P2 




a an «o P 4 




a a 3 



Figure 3. The first sub case 



P2 




a a 3 



Figure 4. The second sub case 



X 




a Y , where X is P2 or P 4 or P-j. 




Figure 5. The second case 
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X 




\P z 



Figure 6. The second sub case 
X 




\P 7,0" 



Figure 7. The first case for Z 

Here and below cc, f3, 7, 5 mean entire images ctj. 
There are three sub cases: 

(1) The second predicate (Y on [Fig. 5]) separates one entire image; 

(2) Y separates patterns belonging to half an image; 

(3) Y doesn't separate either an image or half an image. 

Let us consider the second sub case only (i.e. where Y separates half an image) 
[see Fig. 6]. 

Now we have to consider possible cases for Z. There are two of them. In the first 
one Z separates the rest of image f3 [see Fig. 7]. 

If we try to analyze the situation substituting predicates Pj into X, Y, Z we get 
[Tab. 2] that shows recognition time for all the cases except unrealizable ones. A 
fraction means that half the image is recognized in time shown in the numerator 
and the other half is recognized in time shown in the denominator. 

[Tab. 3] shows the result of comparison those three cases with algorithms A, B, C 
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image\ case 


1 


2 


3 


a 


2 

3 


> 4 


1 


CHi 


1 


■2 

3 


> 4 


OL2 


> 4 


1 


3 


«3 


> 4 


> 4 


> 4 



Table 2. Recognition time for the first case 



case 


1 


2 


3 


A 


+ 


+ 


+ 


B 


+ 


+ 


+ 


C 


+ 


+ 


+ 



Table 3. The result of comparison 



. Sign '+' means that algorithm in the row is better than algorithm in the column. 

So we see there is no algorithm better than A or B or C. Let us find out about 
the second case for Z, i.e. when Z separates an entire image 7 [see Fig. 8]. 

Again we get two tables [Tab. 4] which have the same meaning as [Tab. 2, 3]. 

It gives us the same result. 

We finished the second sub case. The other two sub cases can be investigated 
similarly. 

The theorem is proved. 



There is also a more general theorem on nontransitivity. 

Theorem 2. Let n > 3. Then there are such , S, and Pi that there exist n 
recognition algorithms Aj forming a nontransitive sequence: A c < A! < A 2 < 
. . . « A n _! < Ao. 

Proof. 



image\ case 
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■2 
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■2 
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1 


ati 


2 

4 


3 


1 


1 


■2 
4 


3 


Oil 


3 


1 


'2 
4 


3 


1 


'2 
4 


«3 


> 4 


> 4 


> 4 


> 4 


> 4 


> 4 



case 


1 


2 


3 


4 


5 


6 


A 


+ 


+ 


+ 


+ 


+ 


+ 


B 


+ 


+ 


+ 


+ 


+ 


+ 


C 


+ 


+ 


+ 


+ 


+ 


+ 



Table 4. Comparison time and result for the second case 
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Figure 8. The second case for Z 




Figure 9. Recognition algorithms Aj 
Define Vi = . ^ . 01 jB ■_^JB , 1 < i < n. We introduce n + 1 images as fol- 

i— 1 ra— i 

lows: « = {^1^2 • • • v n }, ai = {v 2 v 3 . . . v n vi}, a 2 = {v 3 v A . . . v n viv 2 },- . ■ , a n _i = 
{v n Vi . . a n = {0...0}. Each image is a subset of 23 n . Iff — |J a ; . 

0<i<n 

S = {ai I < % < n}. Then we define the signs: -P^(x) = a ni+m , where 

x = ai . . . a n a n+ i . . . a 2n On(n-i)+i • • • a n 2 ', < i < n — 1, 1 < m < n. The 

above mentioned recognition algorithms can be defined as shown on [Fig. 9]. 
They have recognition time T(Y,M) shown in [Tab. 5]. 

The theorem statement follows from that table obviously. Note the example from 
theorem 1 can be got from the last theorem when n — 3. 
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M\Y 


A 


Ai 




A n _i 




1 


2 




n 




2 


3 




1 












«n-l 


n 


1 




n — 1 




n 


n 




n 



Table 5. Recognition time of Aj 
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